Capacity Definitions for General Channels with Receiver Side Information

We consider three capacity definitions for general channels with channel side information at the receiver, where the channel is modeled as a sequence of finite dimensional conditional distributions not necessarily stationary, ergodic, or information stable. The Shannon capacity is the highest rate asymptotically achievable with arbitrarily small error probability. The capacity versus outage is the highest rate asymptotically achievable with a given probability of decoder-recognized outage. The expected capacity is the highest average rate asymptotically achievable with a single encoder and multiple decoders, where the channel side information determines the decoder in use. As a special case of channel codes for expected rate, the code for capacity versus outage has two decoders: one operates in the non-outage states and decodes all transmitted information, and the other operates in the outage states and decodes nothing. Expected capacity equals Shannon capacity for channels governed by a stationary ergodic random process but is typically greater for general channels. These alternative capacity definitions essentially relax the constraint that all transmitted information must be decoded at the receiver. We derive capacity theorems for these capacity definitions through information density. Numerical examples are provided to demonstrate their connections and differences. We also discuss the implication of these alternative capacity definitions for end-to-end distortion, source-channel coding and separation. This work was supported by the DARPA ITMANET program under grant number 1105741-1-TFIND. The material in this paper was presented in part at the IEEE International Symposium on Information Theory, Cambridge, Massachusetts, August 1998 and the IEEE International Symposium on Information Theory, Nice, France, June 2007. M. Effros is with the Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125 (email: [email protected]). A. Goldsmith and Y. Liang are with the Department of Electrical Engineering, Stanford University, Stanford CA 94305 (email: [email protected]; [email protected]). April 26, 2008 DRAFT